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Abstract 

Individuals may differ in their parameter values. This article discusses a three-step method of 
studying such differences by calculating and then modeling "individual parameter contributions", 
making the study of heterogeneity in arbitrary structural equation model parameters as technically 
challenging as performing linear regression. The proposed approach allows for a contribution to 
the study of differential error variances in survey methodology that would have been difficult to 
make otherwise. 

1 Introduction 

Structural equation models assume that the same parameter values apply to all individuals. This 
assumption is probably false. Such "aggregated" analyses (Muthen and Satorra 1995) may still 



provide useful models in some cases, particularly when individual differences in parameter values are 
not of substantive interest. In other cases, interest focuses precisely on individual differences in, for 
instance, latent variable means, regressions of one set of latent variables on another, or latent or 
measurement error variances. Of these, it is trivially simple to compare latent variable means across 
values of another variable: a regression of the latent variable on an independent variable will do. 



Most structural equation models involve at least one such regression (e.g. MacCallum and Austin 



2000). Multiple independent variables' effects on the latent mean can be explored just as simply. Less 



simple, however, is the comparison of other types of parameters such as latent regression coefficients, 
variances, and measurement error variances. Such comparisons are, however, often also of scientific 
interest. 

Examples of applications that focus on differences in structural parameters include |Davidov et al.| 



(2008)'s comparison of the regression coefficients of latent attitudes to immigration on latent values 



across 19 European countries, and Kendler et al. (2000 1 's study of heritability of marijuana use in 



twins, which compared variance estimates of additive genetic factors between "use" and "heavy use" 
groups. Individual differences in reliability and validity of survey questions are another example. 



Revilla (2012) split the sample into groups on one covariate at a time and estimated multitrait- 



multimethod models in each group. Scherpenzeel and Saris (1997) used the same approach, as did 



Alwin and Krosnick (1991 pp. 168-172) and Alwin (2007). Instead of looking at the impact of age. 



gender, and education controlling for the other factors, only the marginal impact of each of these 
factors could be assessed ( Revilla 2012[ p. 52) . While it is in principle possible to split the sample by 



the interaction of these three covariates, this would have resulted in groups far too small for estimating 
the multitrait-multimethod parameters of interest. Moreover, to allow for multiple-group structural 
equation modeling, the "age" variable required categorization. 

Solutions to these complications exist. However, relating possibly latent variances, covariances, and 
regression coefficients to explanatory variables is not currently as simple an exercise as regressing an 
outcome on a set of predictors. As demonstrated in the examples, multiple group structural equation 



modeling (Sorbom 1974 Bollen 1989[ [Raykov and Marcoulides 2006), though ffexible and entirely 
general, can lead to a large number of groups, parameters, and equality restrictions. Interaction 



modeling is a possible alternative when differences in regression coefficients are of interest (Kenny 



land Juddj [T9841 [BoIIml [T9951 [Jaccard and mKl p96] [Joreskog and Yang[ p96] [Schumacker and 



Marcoulides 1998 Preacher et al. 2006 van Smeden and Hessen 2013). Interaction modeling has 



the drawback that it is not well-suited to studying differences in variances; and with more than one 



predictor, the dimensions of integration flKlein and Moosbrugger 2000 ) or selection of appropriate 



product indicators or third-order moments (Mooijaart and Bentler , 2010 1 is an additional complication. 



Another possible avenue is multilevel structural equation modeling (Muthen 1989, 1994 Muthen and 



Satorra 1995 ) with random slopes, which has become more convenient with recent software advances 
(Muthen and Asparouhov 20121. This method is flexible but again focuses on slopes rather than 



variances and has the drawback that the random effects models require a normality assumption. 
Random slopes models, moreover, require numerical integration, precluding problems with many 
varying coefficients and a large number of observations. Currently available methods for looking 
at differences in structural equation model parameters are therefore well-developed, but not highly 
convenient for exploration with several candidate covariates, parameter differences, and big data sets. 

This article proposes a three-step procedure that considerably simplifies relating structural equa- 
tion model parameters to covariates. In step one, a single-group structural equation model is fitted. 
In step two, based on the step one estimates, the observed data are transformed into a new data set of 
"individual parameter contributions" (IPC's). These individual parameter contributions can be read 
in by any basic statistics program and regressed on any set of covariates in step three. 

The proposed three-step IPC regression procedure is closely related to the estimation of modifica- 
tion indices and expected parameter changes (Saris et al. 1987). We show that a chi-square test for 



differences in the individual parameter contributions with respect to a grouping variable is equivalent 
to the modification index that would have been obtained in a hypothetical multiple group model with 
equality restrictions. However, the researcher requires only the "aggregated" analysis to obtain the 
individual parameter contributions. Exploring individual differences in arbitrary structural equation 
model parameters thus becomes as simple as running a multiple linear regression or i-test on the 
individual parameter contributions. 

Testing of IPC regression coefficients is also closely related to the econometric literature on "struc- 
tural change" (Kuan and Hornik 1995 Hjort and Koning 20021. "Structural change" tests were 



originally motivated by a search for changes over time in times series model parameters ( Brown et al 
[1975^ Nyblom^ ^1989] |Andrews[ [l993| . |Zeileis| ( [2005J and Zcilcis and Hornik^ (2067\ ge neralized this 



econometric technique to maximum likelihood estimation and Merkle and Zeileis (2013) discussed its 
application to the investigation of measurement invariance. The test obtained from IPC regression 



regression is proportional to a subset of the structural change tests that Hjort and Koning ( 2002 
p. 123) show to have optimal asymptotic power. Whereas the structural change approach places 
emphasis on hypothesis tests, however, IPC regression also estimates the differences' magnitude. In 
other words, when between-group differences are investigated, IPC regression provides not only the 
modification index (Lagrange multiplier, score test), but also the expected parameter change. More- 
over, IPC regression allows for modeling of the parameter differences in the third step: for instance, 
differences with respect to several variables at a time may be evaluated, as the application shows. 
A field of application where IPC regression can be useful is survey methodology: survey researchers 



such as Alwin and Krosnick ( 1991 1 , Scherpenzeel ( 1995 1 , Alwin ( 2007 1 , and Revilla ( 2012 1 have studied 



how survey question reliability varies with respondent characteristics. Past research estimated relia- 
bility, or measurement error variance, with a multiple group structural equation model and compared 
the result across groups, a method that does not permit studying the effects of continuous respondent 
characteristics or the simultaneous evaluation of more than one respondent characteristic at a time. 
We contribute to this field by showing how IPC regression can be applied to regress measurement 
error variance in a "quasi-simplex" model of survey error on respondent characteristics. Effects of Age, 
Gender, and Education level on the measurement error variance in answers to the frequency question 
"how often do you use the internet?" are found, but these effects dissappear when controlling for a 
variable indicating that the respondent never uses the internet. We discuss some possible implications 
for the analysis of frequency questions in surveys. 



The remainder of this article is structured as follows. The data transformation yielding individual 
parameter contributions from an estimated single-group model is derived in the following section. As 
is the case for expected parameter changes and modification indices, regressing individual parameter 
contributions on covariates yields only approximately unbiased estimates. The subsequent simulation 
study therefore not only evaluates the performance of the suggested method, but also compares per- 
formance to that of the commonly applied multiple group structural equation model. The application 
is then discussed: after fitting a four- wave quasi-simplex model to a longitudinal probability sample of 
respondents, we assess individual differences in the measurement error variance of answers to a survey 
question. The final section discusses the results and concludes. 



2 Regressing SEM estimates on covariates 

Intuitively, structural equation model parameters are estimated by combining the observations. Al- 
though the way in which each observation contributes to a parameter estimate is not usually linear, 
it is possible to obtain a linear approximation ( Bentler and Dijkstra[ 19841 to each observation's con- 
tribution. Sampling fiuctuations will mean the contributions differ over observations, but differences 
between parameter contributions may also be due to true ("structural") differences in the parameters 
between observations. This section formally motivates individual parameter contributions and shows 
that performing linear regression of the individual parameter contributions on a covariate is equiva- 
lent to regressing the SEM parameter directly on the covariate, if it were possible to formulate such 
a model. The usual regression coefficient hypothesis tests are valid tests of the relationship between 
a covariate and a SEM parameter. Since the independent variable in the linear regression may be a 
set of dummy variables, the results also apply to comparisons between groups. In that case, as the 



appendix shows, IPC regression is equivalent to calculating versions (Bentler and Chou 1992) of the 



more familiar Expected Parameter Change and Modification Index statistics for differences between 
groups in SEM parameters. 

Given a vector y of p observed variables with population covariance matrix S, a structural equation 
model can be viewed as a covariance structure model S = S(0), where S(0) is a continuously 
differentiable matrix-valued (symmetric and positive definite) function of the vector of parameters 6 
of the model (see, e.g. Bollen[ 1989, for more details). 

Given an observed covariance matrix S, based on a sample of size n of y, the vector of parameters 
d is estimated by minimizing with respect to a discrepancy function F(6) = i^(S,5](0)) of S and 
S(0). This approach encompasses maximum likelihood as well as weighted least squares estimators. 

A key matrix is the hessian matrix V = i drrdcr' ' "^ti^re a :— vech(5]) is the half vectorization of 
S. In weighted least squares estimators, V is a weight matrix determined by the choice of estimator 
(Satorra 1989[ ). To study how parameter values vary as a function of observations, we also define n 
vectors of size p{p + l)/2 

d, :=vech[(y,-y)(y,-y)'], (1) 



(e.g. Satorra 19921, so that d = s. In the case of complex sample data, y should be replaced with a 
consistent (weighted) estimate of the population mean (Muthen and Satorra, 1995 p. 283). 

Of interest are differences in the parameter values across values of a vector of covariates z: it may be 
that the parameter vector Oi varies per observation i due to differences in Zj . Hypothetically, if each 0i 
could be directly observed, studying differences would simply amount to a regression such as the linear 
regression Oi = 'yZ'i- For instance, if z is a dummy-coded grouping variable, differences across groups 
in the structural equation model parameters would be reffected in the regression coefficient vector 7. 
If observations 6i containing sampling error were obtained, sample estimates of the regression model 
could be obtained by performing, for instance, a linear regression 



Oi = 'y'Li + 8i. 
This would yield sample estimates 7 — (zz')^^z0, where :— (^1, ...,0„)'. 



(2) 



In practice is not observed, but it is still possible to approximate 7 by creating a transformed 
data matrix and then performing linear regression on z. From the above, it is clear that 

And from Equation [2] and by applying the implicit function theorem, 

'a2F(S,E(6» 



1 



de 

dz 



dOde' 



l'dF{S,T,{e) 



dOdz' 



From 



Satorra 



(19891, it can be seen that the asymptotic limits of 



deae' 



and^^l^ 

oOoz' 



are A'VA 



and A'V(c'd/yz') respectively, letting A := dcr/dd, which can be obtained from the parameter 
values using the expressions given by Neudecker and Satorra (19911. Given model (pi, 9d/9z' is the 
coefficient in a linear regression. 

The effects of z on parameter values can therefore be obtained as 



7 = (zz') 



Wd, 



(3) 



where the transformation matrix W := — (A'VA)^^A'V. In a given sample, an approximately 
consistent estimate W of W can be obtained by replacing the model parameters that determine A 
and V by their sample estimates. In other words, after transforming the observed data as 



t :=W d, 



(4) 



the 7 coefficient vector can be estimated simply by regressing t on z, or by any other linear model for t 
as a function of z. Since the t values represent each observation's contribution to the linearized sample 
parameter estimate, we will refer to t as "individual parameter estimate contributions" (IPC's). 

The variance estimate of 7 obtained in this way will be consistent as well, and "Wald" or t- 
tests will be valid. Moreover, standard errors and test statistics obtained from the regression of 
t on z will be robust to departures of y from normality. This can easily be seen by recognizing 



1987 



Satorra 



var(0) = n ^(t — t)'(t — t) as the robust parameter estimate covariance matrix (Fuller 

and Bentler[ 1994[). Therefore, the method proposed here is distribution- free. Complex sampling 



designs can be taken into account by taking account of clustering and stratification in the variance 
calculations for the linear regression of t on z, something all standard statistical packages allow the 
user to do. 

In the special case where z is a dummy variable for a grouping of interest, a sample "Wald" test of 
Hq : 7 = is shown in Appendix]^ to be equivalent to calculating a "generalized" modification index 



in a multiple-group structural equation model (Saris et al. 1987 Satorra 1989 section 5). Consider 



the situation where a multiple-group structural equation model is formulated, using z as the grouping 
variable, with cross-group equality constraints on all parameters including the parameter of interest. A 
"modification index" is then calculated for the statistical significance of the change in the parameter of 
interest when freeing all parameter equality constraints (Bentler and Chou 1992|). This modification 
index will equal the "Wald" test for ffo : 7 = in the simple regression of t on z. In addition, 7 will 
equal the corresponding "expected parameter change" (depending on the parameterization). 

Just as for modification indices and expected parameter change coefficients, in deriving approx- 
imate consistency of 7 the assumption is made that the second derivative matrix J := A'VA is 
approximately constant between the null and the true model (Saris et al. 19871. In the case discussed 
here, the relevant assumption is that _Ez[J(0|z)] « J. This assumption will not hold in general, since 
the average conditional parameter estimate is not generally equal to the parameter estimate for a 
completely pooled model (Muthen 1989). However, it will hold approximately for parameters de- 



pending on the covariance matrix such as latent regression coefficients and variances when the sample 
covariance matrix is conditioned on z, i.e. when z is included in the model as a fixed covariate with 
direct effects on all the observed variables. 
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Figure 1: Simulation model. Of interest is the true between-group difference in the factor correlation, 
Pi2. Depending on the condition, there is also a difference of "dif.lambda" in the size of the first 
loading. Between-group differences in the intercepts of j/i - j/4 are always present (not shown for 
clarity) . 



In summary, possible differences in structural equation model parameter estimates can be inves- 
tigated by modeling a transformed data set t as a linear function of z, for instance by OLS linear 
regression or a <-test for differences between groups. This suggests a three-step approach to exploring 
differences in any set of parameters of a structural equation model with respect to a set of covariates: 

1 . Estimate the overall single-group model (preferably introducing z as fixed covariates) ; 

2. From the data and step 1, obtain the transformed data set t; 

3. Regress the individual parameter contributions t for the parameter(s) of interest on the covari- 
ates. 



Appendix |B| provides an R function (get.ipc) that generates the transformed IPC data set (t), 
given a data set and a structural equation model fitted with the lavaan package (R Core Team[ 



2012 Rosseel et al. 2013 1. The transformed data can then be written to a file and analyzed by basic 



statistics software or analyzed directly from within R. An example analysis using these functions is 
given in Appendix [C} 



3 Simulation study 

To evaluate the finite-sample performance of the IPC regression proposed here, we set up a small 
simulation study. The study has two goals: 

1. To evaluate the overall performance of the method in terms of bias, coverage, type I error, and 
power in a simple two-group setting; 

2. To compare performance of the method of taking IPC mean differences with that of the currently 
most applied, and asymptotically most efficient, method: multiple group structural equation 
modeling. 

Figure [1] shows the two- factor model (1 degree of freedom) used for our Monte Carlo simulations. 
A two-group population is generated, with all loadings equal to 1 in group 1, and A21 = A42 = 1, A32 — 



1/2, and An = 1 + dif.lambda in group 2. All error variances in both groups equal var(ej) — 0.8. The 
intercepts of all observed variables equal 1 in group 1 and 2 in group 2. The latent variables rji and 772 
are standardized in both groups and their correlation is set to 0.5 in group 1 and to 0.5+ true difference 
in group 2. We assume that interest focuses on this true difference in latent correlation between groups. 
Multiple group structural equation modeling is compared with IPC regression on a group indicator 
as described above. The two procedures are evaluated under conditions determined by fully crossing 
the following simulation experimental factors: 



• 



• 



The true difference in between-group correlation: {-0.4, -0.2, -0.1, 0, +0.1, 0.2, 0.4}. Note that 
the zero condition corresponds to no effect; 

The difference in loadings between groups: {0, 0.1, 0.2}; 

The sample size per group: rig — {125, 250, 500, 1000}. The total sample size therefore equaled 
n = 2ng = {250, 500, 1000, 2000}. 



For each of the 4 x 7 x 3 = 84 resulting conditions, n observations were drawn from a multivariate 
normal distribution with mean vector and covariance matrix depending on the group. The two 
procedures, IPC differences and MG-SEM, were then applied to the sample data, and this process 
was replicated 1000 times for each condition. 

3.1 Results 

Figure [2] plots the true difference in latent correlations in each condition against the difference es- 
timated using the differences in individual parameter contributions. For easy reference, the black 
45-degree diagonal line corresponds to exact equality of the true and estimated difference. Each point 
corresponds to a particular simulation condition, connected with lines. The lines lie very close to the 
ideal of exact equality. When the true difference is strongly positive or negative, it is slightly underes- 
timated in absolute terms. That is, the difference estimate is slightly biased towards zero for very large 
between group differences in the latent correlation. The differences between the various conditions 
obtained from crossing sample size with loading differences are hardly discernible. This implies that 
the IPC difference procedure provides close to unbiased estimates under all of these conditions. 

Figure [3] shows plots the same quantities as Figure[2]when using multiple group structural equation 
modeling (MG-SEM). The right-hand side displays the results obtained when estimating a correctly 
specified multiple group structural equation model. In this simulation setup, that implies there should 
be no misspecified equality restrictions across groups in the multiple group model. As clearly seen in 
Figure Is] the correctly specified multiple group model provides unbiased estimates under all conditions, 
including those with large differences in latent correlation. The correctly specified MG-SEM procedure 
is therefore superior to the IPC difference procedure, which showed a slight bias towards zero in the 
extreme conditions. 

The left-hand side of Figure [3] demonstrates the bias that occurs when the covariance structure 
parameters are erroneously constrained to equality across the groups (the distance from the diagonal 
to the colored lines). This bias can be considerable. Moreover, when the true difference is zero, 
the misspecified equality restrictions yield a positive latent correlation difference estimate. In other 
words, the researcher will, on average, find a difference in latent correlations when there is none; at 
the same time, when there is a true difference in latent correlations, this difference may be severely 
underestimated. The difference between Figure |2] and the left-hand graph of Figure is striking: after 
all, the IPC difference procedure is based only on the results of a one-group pooled model, which 
implicitly also restricts all covariance structure parameters to equality over groups. Figure [3] shows 
that misspecifications in between-group equality restrictions are less influential in the IPC procedure 
than in multiple group modeling. 

One may also wonder whether, when we simply calculate the difference in IPC scores between 
groups, the usual standard error of this difference is valid and 95% confidence intervals so calculated 
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Figure 2: Bias in the mean difference between groups of fPC for the latent correlation. Different 
conditions are shown as differences in color (loading non-invariance) and point shape (number of 
sample observations). 
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Figure 3: Bias in latent correlation differences obtained from multiple group SEM with misspecified 
equality constraints (left) and correctly specified MG-SEM (right). 
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Figure 4: Coverage of 95% confidence intervals of the mean difference between groups of latent 
correlation's IPC's. Different conditions are shown as differences in color (true latent correlation 
difference) and point shape (loading difference). 



Table 1: Empirical Type-I error rates for a = 0.05. 
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actually cover the true difference in 95% of the replications. Figure [l] shows that this is indeed the 
case. The dotted reference line indicates the ideal, 95% coverage. Under all conditions, the confidence 
interval for the difference in IPC's produced by standard software covers the true difference in latent 
correlations about 95% of the time. 

Next, we examine the performance of the test of the hypothesis that the true between-group 
difference in latent correlation is zero. 

Table [l] shows, for both methods, the number of times the sample hypothesis test of equal latent 
correlations was rejected under the condition true difference = 0: i.e., the empirical Type-I error rates 
using a — 0.05. Under the null hypothesis, this proportion of rejected sample tests should be about 
0.05. It can be seen that both procedures provide adequate control of type-I error under the null 
hypothesis. This holds true for the smaller sample sizes as well as when other parameters do differ 
strongly over the groups. 

When the true difference ^ 0, the rejection rate is the empirical power of the test to detect a 
difference in latent correlation. Figure [5] relates the power of the test to sample size and the true 
size of the difference. The left-hand graph in Figure [5] corresponds to the power using MG-SEM, and 
the right-hand graph to the power using the IPC difference test. The overall median power using 
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Figure 5: Power to reject mean difference test for latent correlation. Different conditions are shown 
as differences in color (true latent correlation difference between groups) and point shape (loading 
non-invariancc) . 

multiple group structural equation modeling is 0.0095 higher than that using IPC difference testing. 
For conditions with a small and medium absolute differences (bottom lines), MG-SEM is somewhat 
more powerful, but with large differences (top lines), IPC difference testing was found more powerful 
for smaller sample sizes. 

Overall, the three-step individual parameter change difference procedure evaluated here performed 
almost as well as correctly specified multiple group structural equation modeling. The advantages of 
the IPC difference procedure in the simple setup discussed here that it is much simpler to apply, and 
that it may be more robust to misspecified equality restrictions than multiple group SEM. 



4 Relating measurement error variance in a survey question 
to respondent characteristics 

Internet usage is often studied in relationship with other variables, for instance in the literature on 



pathological internet use (e.g. Gross 20041 and in studies on social inequalities in internet use (e.g. 
Hoffman and Novak 1998 ) . When studying the relationship between internet use and other variables. 



random measurement error introduced by the self-reports may bias the results downwards (Lord and 



Novick 1968 Fuller 1987 1. Worse, when random measurement error variance differs over groups of 



sex, age, and education, true differences may not only be obscured, but spurious differences may be 



erroneously found (Carroll et al. 2006 1. Therefore important questions are 1) how much measurement 



error variance the answers to internet use questions contain, and 2) whether this measurement error 
variance differs over social groups. We demonstrate how these questions may be answered using 
individual parameter contributions and contribute to the literature on survey errors. 



Table 2: Descriptive statistics for four years' observations of log(hours of internet use at work per 
week + I); n = 2838. Covariates not shown. 
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Figure 6: The quasi-simplex model for four consecutive measurement error-prone measurements of 
internet use in the LISS panel. 



We use 2838 observations from the LISS panel study, a simple random sample of Dutch households. 



For more information on the design of the LISS study we refer to Scherpenzeel (20111, and for the 



freely available original data and questionnaires to http://lissdata.nl/ In this longitudinal panel 
study design, four consecutive measurements of internet usage at work are obtained in the years 2008- 
2011. The log-transform of the number of hours per week the respondent claims to use the internet 
at work is taken to reduce skewness; Table |2] shows descriptive statistics for these measures. 

The first step is to formulate a structural equation model estimating measurement error variance 
in the answers to the internet use question. Table [2] shows that the consecutive measurements do not 
all correlate equally: measurements at a greater distance from each other in time correlate less, so 
that the size of the correlations tapers off towards the lower-left corner of the matrix. This is in line 
with the so-called "quasi-simplex" model, in which internet use is not only measured with error, but 



the true internet use may also change over time ( ,Wiley and Wiley 1970 Alwin and Krosnick 1991 
Alwin 20071. Figure [6] shows the single-group quasi-simplex model formulated for these data as a 
structural equation model. 



Fitting this model to the LISS data with the "robust" maximum likelihood estimator (Satorra 
and Bentler 19941 yields a well-fitting model, with estimates shown in Table [3] The model includes 
the direct effect of self-employment, age, age squared, sex, and education level on each of the four 
observed variables. These regression coefficient estimates are omitted from Table [3] for clarity. The 
error variance parameter is estimated at 0.28 ± 0.01. This error variance estimate corresponds to 
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Table 3: Results of fitting the quasi-simplex model with covariates to the LISS data. Intercepts and 
fixed covariate main effects not shown. Note: Satorra-Bentler chi-square is 1.5 with 2 degrees of 
freedom {p = 0.476, scaling factor = 2.3). CFI = 1.000, RMSEA = 0.016, n = 2838. 





var(Error) 


Latent regressions 




Variances 






/321 /?32 1^43 


Fl 


F2 F3 


F4 


Est. 

s.e. 

Stand. 


0.28 

(0.01) 

0.26 


0.91 0.94 0.94 

(0.03) (0.02) (0.02) 

0.89 0.90 0.92 


0.58 

(0.02) 

0.74 


0.13 0.11 

(0.02) (0.01) 

0.16 0.12 


0.08 

(0.02) 

0.09 



reliability estimates of respectively 0.86, 0.86, 0.87, and 0.88 at the four consecutive years 2008 through 
2011. The overall reliability is therefore high, but not perfect: estimates of the relationship between 
internet use and social background will be affected. 

Another question of interest is whether the measurement error variance differs over groups. Differ- 
ential measurement error can strongly affect relationship estimates (Carroll et al. 2006 1. Furthermore, 



survey methodologists want to know what types of respondents provide higher or lower quality data 
(e.g. Alwin and Krosnick 1991 Scherpenzeel 1995 Alwin p007[ Revilla 20121. Having completed 
step one of fitting a single-group structural equation model, we now obtain each individual's param- 
eter contribution to the error variance parameter estimate (step two): a t score for each respondent. 
Step three is then to perform a multiple linear regression of that score on respondent characteristics 
using standard routines. We employ the Im function in R, but linear regression routines such as those 
in Excel, SPSS, or Stata would serve equally well. The characteristics investigated here are whether 
the respondent is self-employed or not (0/1), the year of birth and its square, sex of the respondent, 
and education level - primary, lower secondary (VMBO), middle secondary (MBO), upper secondary 
(HAVO/VWO), lower tertiary (HBO), and higher tertiary (University). 

The multiple linear regression estimates of the relationship between respondent characteristics and 
error variance are shown as effect plots in Figure [7] The estimates themselves can be found in the 
appendix. Figure [7] shows that, controlling for the other characteristics, three of the four respondent 
characteristics (age, sex, and education) have a strong and statistically significant effect on the error 
variance, and therefore on reliability, which is inversely related to the error variance. The reliability 
of internet use self-reports increases with age, decreases with education, and is lower for men than for 
women. 

To survey methodologists and those familiar with the literature on reliability of survey questions, it 
may seem surprising that reliability would be lower for the young and those of a higher education level. 
After all, cognitive functioning is higher for these groups and expected to increase the reliability of 



answers (Krosnick 2011 1. There may, however, be a simple explanation for this finding: the variance 



of measurement errors in self-reports of the number of hours spent using the internet at work may 
simply be related to internet use itself. If one does not use the internet at all, presumably there would 
be little to no measurement error. A possible explanation for the finding that young and educated 
respondents appear to provide lower-quality answers is that these respondents simply tend not to be 
the kind of respondents who use the internet at work for zero hours per week. 

The statistically significant effects on error variance of these respondent characteristics may there- 
fore be mediated by whether or not the respondent never uses the internet at work. As an indicator 
of this variable, we construct a new dummy variable that equals one if the respondent has answered 
"zero hours" on all four occasions and zero otherwise. We then formulate a new structural equation 
full mediation model in which the error variance IPC is influenced by the "all-zeroes" variable, and 
this variable, in turn is influenced by the four covariates. This full mediation model fits the data well 
(xlji = 11, df = 9, p = 0.276, RMSEA = 0.009; full results can be found in the appendix). Thus, 
it appears that measurement error variance indeed differs over these respondent characteristics, but 
only for the reason that respondents with these characteristics tend not to be those who never use 
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Figure 7: Estimated conditional differences in measurement error variance by respondent character- 
istics: effect plot for the regression of the IPC of the error variance parameter on self-employed, year 
of birth, gender, and education. 

the internet, something that tends to strongly reduce the error variance: the difference is 0.50 ± 0.03 
corresponding to a fully standardized effect of "all-zeroes" on the error variance of 0.3. 

An evaluation of the IPC regression results can be made by comparing the IPC mean differences 
with the results that would have been obtained by separate multiple group analyses. It is not possible 
to formulate a multiple group structural equation model in which the effect of some grouping variables 
on the error variance are controlled for the effect of other grouping variables. However, it is possible to 
repeat the IPC regression (the third step) , but this time regressing the IPC on only one covariate at a 
time. Since there are four covariates, this yields four linear IPC regressions. This analysis can then be 
compared with four separate multiple group analyses in which the groups are formed by each of the 
four covariates (no between-group restrictions are imposed). We may then compare the differences in 
the error variance estimates over groups obtained from each of the four multiple group analyses with 
those obtained from the four simple IPC regressions. 

Table |4] shows that the estimated differences in error variance between the groups obtained by 
either multiple group structural equation modeling or regression of the individual parameter contri- 
butions on a covariate are very close to each other. Since multiple group SEM is an asymptotically 
unbiased procedure, we have labeled the difference between the two sets of estimates "bias" (fourth 
column of Table El. Overall, the bias is small. Thus, in this example, if a single grouping variable 
is of interest, the results obtained by multiple group SEM are almost identical to those obtained by 
simply assessing between-group differences in IPC's. Multiple group SEM does not, however, allow 
for multiple grouping variables whose shared variance is to be partialed out. 



12 



Table 4: Second column: estimates of between-group difference in error variance parameter obtained 
by taking a difference in fPC's; third column: difference estimated by fitting a multiple group SEM 
in each group and calculating the difference in the maximum-likelihood estimates. 

fPC regression Multiple group SEM Bias 



Female — male 




0.0576 


Self-employed — 


not 


0.09f9 


Education — Primary 




VMBO 




0.f575 


MBO 




0.3f22 


HAVO/VWO 




0.227f 


HBO 




0.3f4f 


WO 




0.4f33 



0.0627 


-0.0051 


0.0939 


-0.0020 


0.f638 


-0.0064 


0.3f96 


-0.0051 


0.2219 


-0.0051 


0.3225 


-0.0051 


0.4420 


-0.0051 



5 Discussion and conclusion 

We suggested to study differences in structural equation model parameters over groups by a simplified 
three-step procedure. In the first step a single-group SEM is fit to the data; in step two the trans- 
formation discussed in section [2] is applied to the data to obtain individual parameter contributions 
(IPC's); and in step three, these IPC's are related to grouping variables by standard methods such 
as dummy regression or i-tests. More generally, we showed that performing a linear regression of the 
IPC's on a set of covariates, categorical or continuous, amounts to calculating misspecification-robust 
"expected parameter changes" and "modification indices" for the heterogeneity of an arbitrary SEM 
parameter estimate with respect to the covariates. The procedure is simple to apply, since after 
step two it amounts simply to regressing a dependent variable, the IPC for a parameter, on a set of 
covariates explaining heterogeneity in that parameter. 

A small Monte Carlo simulation evaluated the performance of this method in a particular setup, 
namely when interest focuses on between-group differences in latent factor correlations in a two-factor 
model. The IPC regression method performed excellently in this setup, showing little bias and good 
coverage properties. Moreover, we compared the IPC regression method in this setup to the more 
common, and theoretically most efficient, multiple group SEM. There were few differences between the 
two methods in bias, variance, type I error control, or power. In our application of the IPC regression 
method to between-group variations in error variances, no differences larger than 8% between the IPC 
regression method and multiple group SEM were found. 

Naturally, there are many possible structural equation models, parameters of interest, and hetero- 
geneity setups; no single simulation study can evaluate the performance of this method under all of 
them, nor was that our goal. Still, this represents a limitation of the present simulation study. Our 
theoretical results as well as the correspondence between the empirical results in the application to 
the quasi-simplex model suggest that the method may show good properties in other situations as 
well, although a larger range of situations remains to be investigated. Exceptions may occur when the 
base single-group model is extremely misspecified, something that may occur in certain models (e.g. 
growth models) when possible mean differences with respect to the covariates are not included in the 
single- group model. For this reason we advise including the covariates' main effects in the single-group 
analysis before estimating the individual parameter contributions. 

Using four consecutive waves from a longitudinal panel study of Dutch households, we estimated 
measurement error variance in self-reports of internet use. Individual parameter contributions to 
the measurement error variance varied over age, gender, and education level. Contrary to what 
might be expected, however, young people and the highly educated tended to provide lower quahty 
information. We hypothesized an explanation for this counterintuitive finding: that older people, the 
lower-educated, and women tend more often to simply never use the internet, a true value that would 
not easily incite response errors. A mediation model in which the variable "answering zero hours on 
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all four measurement occasions" mediated the effect of the respondent characteristics on individual 
parameter contributions to the error variance was in line with that hypothesis. After taking into 
account that those who always use the internet for zero hours per week have a lower measurement 
error variance, there was no further effect of respondent characteristics on the error variance. The 
finding that measurement quality in a frequency question is related to the frequency itself corresponds 
to findings by Tourangeau et al. ( 2000 1 and Saris et al. ( 2008 1 . This finding could not be explained 



by differences in respondent characteristics. Moreover, our study shows that, because of this effect, 
there is differential measurement error in internet use self-reports with respect to age, gender, and 
education level. Therefore researchers explaining or predicting internet use should be careful to allow 
for this differential measurement error so as to prevent bias. 

Extensions to the suggested use of IPC's are trivial but require evaluation as to their efficacy. 
For example, clustering of IPC's may be an attractive robust method of modeling heterogeneity in 
parameters without allowing the clusters to be determined by heteroskedasticity and mean differences. 
The method may also be used to investigate measurement invariance, in which case a comparison with 



the test statistics suggested by Merkle and Zeileis (20131 would be an interesting avenue of further 



study. It may be possible to model derived parameters, such as standardized coefficients and indirect 
or total effects in mediation analysis. Categorical data and "extended structural equation models" 
such as latent class analysis were not discussed here; both could in principle be accomodated by 



adopting a framework such as that of Muthen (20021 or du Toit (2012). Finally, we did not deal with 



missing data. Here, one avenue would be casewise gradients with respect to the FIML fitting function, 
and another approach might be to simply delete missings "pairwise" in the multiplication W • d. How 
these approaches perform in practice, and how they perform relative to the currently best available 
methods, remains to be investigated. 



In summary, individuals may differ in their parameter values. This article discussed a considerable 
simplification and extension of the study of arbitrary SEM parameter heterogeneity with respect to 
covariates. R code implementing this idea for the SEM package lavaan is given in the Appendix, as 
well as an example application. Using individual parameter contributions, we were able to provide 
a contribution to survey methodology that would have been difficult to make otherwise. We hope 
that other structural equation modelers will also find the suggested method a useful tool to study 
individual differences in SEM parameters. 
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A A difference in IPC's and the corresponding test can be in- 
terpreted as an EPC and MI in a multiple group structural 
equation model 

The expected parameter change for freeing an equality-constrained parameter 9g in group g is defined 



as ( [Saris et al.[|1987[ ): 

d^F \ fdF 



Note that the hcssian with respect to the entire parameter vector is used, not just with respect to the 
parameter of interest. This takes account (to some degree) of expected changes in other, correlated, 
parameters due to freeing the constraint ([Sorbom 1989j). The EPC's can be used to obtain an estimate 



of the difference between groups from multiple group SEM as the difference of the EPC's 5g for each 



group (Bentler and Dijkstra 1984 1: {6 + 82) - {9 + 5i) =. 82 - 5i 



Theorem 1. Formulate an MG-SEM with equality restrictions on all parameters. Order groups by 
size of 9g. Define t as in ijV. Then: (52 — i5i = t2 — ti. 
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Proof. The estimated difference between groups from MG-SEM is: J2^{dF/d62) - J^\dF/d9i) = 
J^^[{dF/d92) — {dF/dOi)] because Ji — J2 since all parameters estimates are equal and 9i and 62 
play the same role in different groups: Ag^ = Ag^ and Vi = V2 so Ji = J2 — J- 

Now, dF/dOg = A'V[sg 
A'V[s2 ■ 



(e.g. [Neudecker and Satorra|p9T] ). So {dF/de2) - {dF/dOi) = 

A'V[d2 - di] 



si], again because a{92) = o-{9i)- Since by definition Sg = dg, A'V[s2 — si] 
leading to 62 - 5i = J-^A'V[d2 - di] = {2 - ti. 



a 



The "modification index" (score test) for freeing this restriction is defined as the difference divided 
by the variance of the difference. So me standard so ftware packages currently use the inverse hessian 
J~^ as the variance. However, from jSatorra (1989 section 5) it is clear that this is only valid when 
asymptotically optimal (AO) estimation has been used. For non-AO estimation, such as pseudo- 
maximum likelihood or in the case of nonnormally distributed y, a sandwich estimator is needed 



(Satorra 1989 1. The resulting hypothesis test is then equivalent to the so-called "generalized" score 

test (Boos 1992). From t2— ti = J~^A'V[d2 — di], it is clear that a standard z-test performed on the 
difference in IPC's fulfills this requirement since it will automatically calculate the sandwich variance 
var(^2 — Si) — J~^A'Vva.i{d2 — di)VAJ~^. Thus, squared z-statistics from the regression of t on a 
group indicator can be interpreted as generalized score tests that are robust to nonnormality. 

Complex sampling can be taken into account as described in the text: by adjustin g d using the 
weights and estimating va.i{d2 — di) incorporating clustering and stratification identifiers (Muthen and 



Satorra 1995 1. In general MG-SEM can be expected to yield the most accurate results. However, since 



some SEM software does not correct the modification indices for nonnormality or complex sampling, 
in cases of strong nonnormality or complex sampling design effects, the proposed procedure may 
therefore actually be preferable to MG-SEM. 



B R code providing the individual parameter contributions 



# Obtain the derivative of SEM parameters w.r.t. sigma 
get.g <- functionCf it) { 

# d sigma / d theta 

D.free <- lavaan: : : computeDelta(f itOModel) [[1] ] 
colnames(D .f ree) <- names(coef (f it)) 

# NT weight matrix 

V.nt <- lavaan: : :getWLS.V(fit) [[!]] 

H <- solve (t (D.free) "/.*•/ V.nt "/.*•/ D.free) 
g <- H "/.*•/ t (D.free) "/.*•/ V.nt 



# Obtain the rescaled individual moment contributions, 
get.di <- function(x, center=NULL) { 

if (is .null(center) ) center <- colMeans(x, na.rm=TRUE) 

xd <- t(t(x) - center) 

p <- NCOL(xd) ; n <- nrow(xd) 

idx.vech <- vech(matrix(l : (p~2) , p)) # Remove redundant elts. 

a <- matrix (rep (xd, p) , ncol = p * p) [, idx . vech] 

b <- xd[, rep(l:p, each = p)] [, idx .vech] 

d <- a * b * n/(n - 1) 



cbind(x, d) # add means 
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# Obtain the IPC dataset given 

# a matrix of observations X and a lavaan fit object, 
get . ipc <- fimctionCX, fit) { 

g <- get.g(fit) 

d <- t(get.di(X)) 

wi <- as .data, frame (-t(g %*'/„ d) ) # assumes meanstructure=TRUE. 



wi 



} 



C Example application 



library (lavaan) 

library (systemf it) # Convenient for regression with many dep. variables 

# The Holzinger and Swineford (1939) example, single-group model 
HS. model <- " visual =- xl + x2 + x3 

textual =- x4 + x5 + x6 
speed =- x7 + x8 + x9 " 

# Fit the model 

fit <- lavaan(HS. model, data=HolzingerSwinef ordl939, 
auto. var=TRUE, auto. fix. f irst=TRUE, 
auto .cov.lv. x=TRUE , 
meanstructure=TRUE, int . ov. free=TRUE) # necessary 

# Obtain the IPC's and join them with the original data 

ipc. data <- cbind(HolzingerSwinef ordl939, get . ipc(f itODataSXC [1] ] , fit)) 

# Use systemf it for convenience to regress parameters on covariates. 
equation. list <- lapply(names(coef (f it)) , function(parname) { 

as .f ormula(sprintf (" 'VoS ' - sex+ageyr+agemo+school+grade" , parname)) 
}■) # Generates list of formulas 
names (equation. list) <- names(coef (f it)) 

fit. system. ols <- systemf it (equation. list , data=ipc .data) 
summary(f it . system. ols) 

# Grade appears to be an important grouping variable 



19 



Table 5: Coefficient estimates for regression of the IPC of the error variance parameter on respondent 
characteristics (n == 2838). The R^ is 0.03. 





Estimate 


s.e. 


t-value 


p- value 


(Intercept) 


0.08 


(0.09) 


0.93 


0.35 


Self-employed 


0.09 


(0.06) 


1.40 


0.16 


Year birth 


6.71 


(0.91) 


7.34 


0.00 


Year birth^ 


-2.33 


(0.91) 


-2.55 


0.01 


Female 


-0.08 


(0.03) 


-2.42 


0.02 


Education 










Primary school 


0.00 








VMBO' 


0.17 


(0.09) 


1.90 


0.06 


MBO 


0.26 


(0.09) 


2.82 


0.00 


HAVO/VWO 


0.18 


(0.10) 


1.83 


0.07 


HBO 


0.29 


(0.09) 


3.21 


0.00 


University 


0.37 


(0.11) 


3.54 


0.00 



D Internet use study results 

Extended results for the application of IPC regression to the internet use example are given in this section. 
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Table 6: Full mediation SEM: error variance IPC (dependent variable), "all-zeroes" indicator (media- 
tor), and covariates (predictors). Satorra-Bentler chi-square is 11 with 9 degrees of freedom (p ~ 0.276, 
scaling correction = 0.953); CFI = 0.999, RMSEA := 0.009, n = 2838. 



Parameter 


Est. 


s.e. 


z 


Regression coefficients 








Error variance IPC on 








All-zeroes 


-0.50 


0.03 


-17.0 


All-zeroes on 








Self-employed 


-0.06 


0.03 


-2.4 


Year birth 


-10.07 


0.40 


-25.1 


Year birth^ 


6.46 


0.41 


16.0 


Female 


0.10 


0.02 


6.2 


Education: VMBO 


-0.13 


0.04 


-3.1 


Education: MBO 


-0.31 


0.04 


-7.4 


Education: HAVO/VWO 


-0.35 


0.04 


-7.8 


Education: HBO 


-0.46 


0.04 


-11.3 


Education: University 


-0.51 


0.04 


-11.4 


Variance parameters 








Error variance IPC 


0.75 


0.06 


11.8 


All-zeroes 


0.17 


0.00 


50.1 
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